Repository navigation
feat(rdma): add RC connections and notification-driven event loop - #40
Merged
thweetkomputer merged 5 commits intoNov 17, 2026
Merged
Conversation
|
Important
This repository does not receive automatic reviews because it has fewer than 10 stars.
|
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Add worker-owned RC listeners and connections so an idle worker can accept and establish RDMA connections through its existing io_uring. The optional RDMA build now supports Listen/Accept/Connect/Close, connection deadlines, and CM private data.
Implementation
Ownership
RC Engine, Device, Listener and Connection state uses non-atomic LocalSharedPtr/LocalWeakPtr. Strong references still retain states across suspended coroutines; weak back-references avoid cycles and detect owner expiration. Handle access, movement and destruction, including coroutine-frame destruction after shutdown, stay on the creating thread. Debug builds check affinity.
Bycorf had no local reference-counting utility; the implementation follows Lavik's LocalSharedPtr thread contract and adds weak observation and incomplete-type support without a dependency on Lavik. The audit retains std::shared_ptr for cross-thread timer cancellation, TLS context sharing/state handoff, runtime startup coordination, and exported PD/MR/verbs-context ownership.
Validation
Latest ownership revision on ARM64:
ctest --test-dir /home/ubuntu/.cache/bycorf-localrefs/default --output-on-failure— 118/118 passed./home/ubuntu/.cache/bycorf-localrefs/rdma/bycorf_rdma_rc_fake_tests— 24/24 passed./home/ubuntu/.cache/bycorf-localrefs/asan/bycorf_unit_tests --gtest_filter='LocalSharedPtr*'— 7/7 passed;/home/ubuntu/.cache/bycorf-localrefs/asan/bycorf_rdma_rc_fake_tests— 24/24 passed.The tests cover strong/weak lifetime, self-weak release during destruction, incomplete types, construction failure, foreign-thread misuse assertions, real worker-thread shutdown, lazy Accept after handle destruction, CM/CQ failure fanout and cancellation retirement. Earlier revisions passed 6 Soft-RoCE RC tests and the registration integration test; the current host has no Soft-RoCE device, so physical-provider checks were not rerun for this revision.
Scope and limitations
Numeric IPv4/IPv6 addresses only; CM private data is limited to 56 bytes. Data-posting APIs, automatic CQ growth, and application buffer pools are outside this change. RDMA remains disabled by default and independent of DPDK/SPDK. Real-device validation used Soft-RoCE; physical RNIC providers have not been tested.